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AN ULTRASOUND IMAGING SYSTEM AND 
METHODS OF IMAGING USING THE SAME 

Field of the Invention 

[0001] The present invention relates generally to imaging systems and, 

5 specifically, to an ultrasound Imaging system and methods of imaging using the 
same. 

Background of the Invention 

[0002] Ultrasound-guided interventional procedures such as breast 
biopsies and prostate brachytherapy are well-known. Needles can be inserted 

10 into the body and either obtain a biopsy sample or deliver a dose of a selected 
therapy. For biopsies, it is desirable to target a specific volume when obtaining 
a tissue sample. Where a dose is being administered to a target volume, it is 
desirable to track the precise location of the needle delivering the dose in real- 
time to ensure that the therapy is delivered according to plan. 

1 5 [0003] Radioactive seeds can be used as a therapy to treat tumors in 

prostates. In order to ensure adequate coverage of the therapy, it is desirable to 
implant the seeds a pre-determined distance apart. If the distance between the 
seeds is too large, tissue between the seeds may not receive the amount of 
therapy needed for the treatment. If, instead, the seeds are too closely 

20 positioned, the tissue can be over-exposed. Further, It is desirable to ensure 
that the implantation of the seeds is limited to the target volume in order to 
prevent the therapy from adversely affecting othenA^ise healthy tissue. 
[0004] In robotic-aided interventional procedures, such as robot-aided 

and ultrasound-guided prostate brachytherapy as well as free-hand ultrasound- 

25 guided biopsy procedures, a needle is inserted free from parallel trajectory" 
constraints. Oblique insertion of the needle, however, can result in the needle 
intersecting the two-dimensional ("2D") trans-rectal ultrasound ("TRUS") Image 
and appearing as a point, leading to blind guidance. \ 
[0005] Some investigators have developed automatic needle 

30 segmentation methods to locate needles for biopsies and therapy. These 
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methods, however, require that the needle be completely contained in the 2D 
ultrasound ("US") image, 

[0006] The general operation of ultrasound transducers has provided 
less-than-deslrable image resolution in some instances. Image quality for less 
5 significant regions distal from the target volume or even along the shaft of the 
needles may not be as critical as for the region surrounding the needles. This is 
especially true for therapy where seeds are being implanted in a target volume. 
Current ultrasound techniques, however, are directed to the capture of generally 
evenly distributed images, regardless of the content of the volume targeted by 
10 the images, 

[0007] It is, therefore, an object of the present invention to provide a 

novel method of imaging using an ultrasound imaging system. 

Summary of the Invention 

[0008] In an aspect of the invention, there is provided a method of 

15 registering the position of an object moving in a target volume in an ultrasound 
imaging system, comprising: 

capturing a first ultrasound image of a target volume; 
capturing a second ultrasound image of said target volume after 
said capturing of said first ultrasound image; and 

20 identifying the position of said object in said target volume using 

differences detected between said first and second ultrasound images. 
[0009] In a particular aspect, a difference map of the differences between 
the first and second ultrasound images is generated. The difference map can 
be thresholded to identify significant changes between the first and second 

25 ultrasound images. In another particular aspect, the object is a needle, and the 
difference map is filtered to identify voxels in the difference map corresponding 
to a characteristic of the needle. In a further particular aspect, the first 
ultrasound image is captured prior to entry of the object in the target volume. 
[0010] In another aspect of the invention, there is provided an ultrasound. 

30 imaging system for registering the position of an object moving in a target 
volume, comprising: 
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a transducer for capturing a first ultrasound image and a second 
ultrasound image of a target volume; and 

a processor for detecting differences between said first and 
second ultrasound images to Identify the position of said object in said target 
5 volume. 

[001 1] In a particular aspect, the processor generates a difference map 
from the first and second ultrasound images identifying the differences 
therebetween. The processor can threshold the difference map to identify 
significant differences between the first and second ultrasound images. 

1 0 [0012] In a further aspect of the invention, there is provided a method of 

imaging using an ultrasound imaging system operable to capture image data 
from a target volume, comprising: 

detemnining a region of interest in the target volume; 
determining a segment of an operational scan range of a 

1 5 transducer of said ultrasound imaging system encompassing said region of 
interest; and 

focusing said ultrasound imaging system on said segment of said 
operational scan range during Image capture. 

[0013] In a particular aspect, the region of interest is an area of expected 

20 activity of an object. In another particular aspect, the object is a needle, and the 
region of interest includes the area along a trajectory of the needle beyond a tip 
of the needle. In a further particular aspect, the determining of the region of 
interest includes the expected position of a needle in the target volume. The 
transducer can be, for example, a rotational transducer. In still other particular 
25 aspects, the focusing includes capturing image data in the segment of the 

operational scan range at a greater scan density than outside of the segment of 
the operational scan range, or capturing image data only in the segment of the 
operational scan range. 

[0014] In a still further aspect of the invention, there is provided an 
30 ultrasound imaging system, comprising: 

a transducer for capturing ultrasound images of a target volume; and 
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a processor for determining a region of interest in the target volume, for 
determining a segment of an operational scan range of said transducer 
encompassing said region of interest, and for directing said transducer to 
focus on said segment of said operational scan range, 
5 [0015] In a particular aspect, tlie processor determines an area of 

expected activity to determine the region of interest. In another particular 
aspect, the transducer is a rotational transducer and the processor determines 
an angular sector of the operational scan range of the rotational transducer. In a 
further particular aspect, the processor directs the transducer to capture image 
1 0 data in the segment of the operational scan range at a greater scan density than 
outside of the segment of the operational scan range. In a still further particular 
aspect, the processor directs the transducer to capture image data only in the 
segment of the operational scan range. 

[0016] The invention enables the position of the needle to be accurately 

1 5 determined. By"onlyanaryzirig"iF^^ data"that varies significantly between two 
ultrasound images, the needle can be readily differentiated from complex 
backgrounds in the ultrasound images. Further, by focusing on a segment of 
the operational scan range of the transducer of the ultrasound imaging system 
during image capture, more detailed image data can be captured around the 
20 needle to enable its position to be determined with a desired level of accuracy. 
This can be achieved without sacrificing the scanning speed in some cases. 

Brief Descrf ption of the Drawings 

[0017] Embodiments will now be described, by way of example only, with 

reference to the attached Figures, wherein: 
25 Figure 1 is a schematic diagram of an ultrasound imaging system 

for imaging a target volume in a subject; 

Figure 2 shows a three-dimensional ("3D") TRUS transducer 

forming part of the ultrasound imaging system of Figure 1 capturing a set of 2D 

US images of a needle; 
30 Figure 3 is a flow chart of the general method of operation of the 

system of Figure 1 ; 
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Figure 4 shows a reconstructed 3D image generated from 2D 
ultrasound images captured by the TRUS transducer shown in Figure 2; 

Figure 5 Is a flow chart illustrating the method of perfonning a 

. subsequent 3D US scan; 
5 Figure 6 is a sectional view of a scan range corresponding to a 

region of interest determined using the method of Figure 5; 

Figure 7 is a flow chart that illustrates the method of segmenting a 

needle; 

Figure 8 is a flow chart that illustrates the method of determining 
10 the greyscale-leve! change threshold; 

Figure 9 is a flow chart that illustrates the method of generating a 

difference map; 

Figures 10a and 10b show the difference map generated using 
the method of Figure 9 before and after pre-filtratlon respectively; 
1 5 Figure 1 1 is a flow chart that illustrates the method of performing 

regression analysis; 

Figure 12 is a flow chart that better illustrates the method of 

filtering the difference map; 

Figure 13 shows the difference map of Figures 10a and 10b 
20 immediately prior to the performance of the final regression analysis; and 

Figures 14a to 14c show various 2D US images generated using 
the ultrasound imaging system of Figure 1 . 

Detailed Description of the Embodiments 

[001 8] The method of registering the position of an object such as a 
25 needle provides for the near real-time identification, segmentation and tracking 
of needles. It has a wide range of applications, such as biopsy of the breast and 
liver and image-guided interventions such as brachytherapy, cryotherapy, as 
well as other procedures that require a needle or needles to be introduced into 
soft tissues and be positioned accurately and precisely. The use of the method 
30 is described in robot-aided 3D US-guided prostate brachytherapy for the 
purpose of illustration. 
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[0019] Transperineal prostate brachytherapy provides an improved 
alternative for minimally-invasive treatment of prostate cancer. Pubic arch 
interference ("PAI") with the implant path, however, occurs in many patients with 
large prostates and/or a small pelvis. These patients cannot be treated with 
5 current brachytherapy using parallel needle trajectories guided by a fixed 

template, because the anterior and/or the antero-lateral parts of the prostate are 
blocked by the pubic bone, 

[0020] To solve the PAI problems, it is desirable to free needle insertions 

from parallel trajectory constraints. Oblique trajectories allow patients with PAI 

10 to be treated with brachytherapy without first undergoing lengthy hormonal 
downsizing therapy. In addition, changes in the prostate size prior to 
implantation, where the therapy is determined in advance of the procedure, and 
during the implantation, due to swelling of the prostate, may require re- 
optimization of the dose plan. The combination of precision 3D TRUS imaging, 

1 5 dosimetry and oblique needle insertion trajectories can provide the tools needed 
for dynamic re-optimization of the dose plan during the seed Implantation 
procedure by allowing dynamic adjustments of the needle position to target 
potential "cold spots". Cold spots are areas more than a desired distance from 
seed implantation locations, resulting in less-than-desired exposure. Further, 

20 the dosimetry can be dynamically adjusted to compensate for deviations in the 
actual needle trajectories or shifting in the target volume. 
[0021] A 3D TRUS-guided robot-aided prostate brachytherapy system is 

shown generally at 20 in Figure 1 . The system 20 includes a TRUS transducer 
24 coupled to a motor assembly 28 that operates to control the longitudinal 

25 movement and rotation of the TRUS transducer 24. The TRUS transducer 24 is 
also coupled to a conventional ultrasound machine 32 for displaying image data 
as it is captured by the TRUS transducer 24. A video frame-grabber 36 Is 
connected to the ultrasound machine 32 to capture image data therefrom. The 
video frame-grabber 36 preferably operates at 30Hz or greater to provide rapidly 

30 updated ultrasound images. 

[0022] A computer 40 is connected to the video frame-grabber 36 and 
retrieves ultrasound images from the memory of the video frame-grabber 36. 
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The computer 40 is coupled to a mover controller module ("MCM") 44 that is 
coupled to and controls the motor assembly 28. The computer 40 is also 
connected to the TRUS transducer 24. Further, the computer 40 is connected 
to a robot 48 having a needle driving assembly 52 and needle guide 56 for 
5 controlling movement of a needle 60. The needle 60 is used to deliver therapy 
to a prostate 64 of a patient The robot 48 receives needle control commands 
from and transmits needle position information to the computer 40, 
[0023] The TRUS transducer 24 is operable to continuously capture 

radial 2D US images over a radial operational scan range. The MCM 44 which 

1 0 controls the TRUS transducer 24 is in communication with the computer 40 to 
receive TRUS control commands via the serial port of the computer 40. The 
TRUS control commands direct the MCM 44 to control the motor assembly 28. 
In turn, the motor assembly 28 controls the longitudinal movement and rotation 
of the TRUS transducer 24. Additionally, the TRUS control commands control 

1^5 "the tirning'of rnfiage'data capture of the TRUS transducer 24. — 

[0024] The needle driving assembly 52 Includes a robotic arm with six 
degrees-of-freedom. The degrees-of-freedom correspond to translations of the 
needle 60 in three dimensions and rotation of the needle 60 about three 
orthogonal axes. In this manner, the needle 60 can be positioned in a wide 

20 variety of orientations. The needle guide 56 is a one-holed template that is used 
to stabilize lateral movement of the needle 60 during insertion. 
[0025] The computer 40 is a personal computer having a processor that 
executes software for performing 3D image acquisition, reconstruction and 
display. The processor also executes software for determining dosimetry of a 

25 selected therapy, and for controlling the TRUS transducer 24 and the robot 48. 
The software executed by the processor Includes TRUS controller software, 
positioning software, imaging software, 3D visualization software and dose 
planning software. 

[0026] The TRUS controller software generates TRUS control commands 
30 for directing the MCM 44, thereby controlling the longitudinal and rotational 
movement and the image data acquisition timing of the TRUS transducer 24. 
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[0027] The positioning software generates needle control commands to 
control movement of the needle driving assembly 52 of the robot 48. The 
positioning software can direct the robotic ann to move In terms of world or tool 
coordinate systems. The world coordinate system is fixed to the ground, 
5 whereas the tool coordinate system is fixed to the robotic arm. Further, the 
positioning software can direct the needle driving assembly 52 to control the 
longitudinal movement of the needle 60. 

[0028] The imaging software captures, analyzes and processes 

ultrasound images using the image data retrieved from the memory of the video 
10 frame-grabber 36. The positioning software provides needle position 

information using the selected coordinate system. In turn, the imaging software 
directs the TRUS controller software to vary the operation of the TRUS 
transducer 24 as will be explained. 

[0029] The 3D visualization software renders 3P images to be presented v 

15 on a display (not shown) of the computer 40 using the Image data captured and 
processed by the imaging software. In particular, the 3D visualization software 
generates three orthogonal views of the target volume: two that are co-planar to 
the needle 60 and a third that generally bisects the needle 60. 
[0030] The dose planning software performs precise image-based needle 

20 trajectory planning. In addition, the dose planning software provides planned 

needle trajectory information to the 3D visualization software so that the planned 
needle trajectory can be overlaid atop the US images on the display. The actual 
needle trajectory can then be viewed in relation to the planned needle trajectory. 
The dose planning software can also receive and process the US images from 

25 the imaging software and dynamically re-determine the dosimetry based on the 
actual needle trajectory and seed implantation locations. 
[0031] . Prior to use, the positioning software controlling movement of the 
robot 48, the needle driving assembly 52 and, thus, the needle 60, and the 
imaging software are calibrated. During calibration, the mapping between the 

30 selected coordinate system of the positioning software and the 3D TRUS image 
coordinate system is determined and synchronized. In this manner, the imaging 
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software can be made aware of the expected position of the needle 60 before 
detection via Imaging. 

[0032] By unifying the robot 48, the TRUS transducer 24 and the 3D 
TRUS image coordinate systems, the position of the template hole of the needle 
5 guide 56 can be accurately related to the 3D TRUS image coordinate systeni, 
allowing accurate and consistent insertion of the needle via the hole into a 
targeted position in a prostate along various trajectories Including oblique ones. 
Further, the operation of the TRUS transducer 24 can be varied to focus its 
attention on the expected position of the needle 60. 

10 [0033] Figure 2 shows-the 3D TRUS transducer 24 capturing a set of 2D 

US images. As the TRUS transducer 24 is rotated by the MCM 44, it captures 
image data to generate a series of 2D Images 68. The 2D images 68 are 
captured at generally regular Intervals during rotation of the TRUS transducer 
24. Initially, the TRUS transducer 24 captures a 2D image 68 every one degree 

15 of rotation and rotates through 1 00 degrees, thereby capturing one hundred and 
one 2D images 68. The captured 2D images 68 are fanned radially in relation to 
the TRUS transducer 24. The needle 60 Is shown having an oblique trajectory 
in relation to the 2D Images 68, and intersects two or more of the 2D images 68. 
[0034] As will be understood, insertion of the needle 60 along an oblique 

20 trajectory results in the intersection of the 2D TRUS image planes. As a result, 
the needle 60 only appears as a point in the captured 2D US images. 
[0035] A near real-time method 100 for identification, segmentation and 

tracking of needles will now be described with reference to Figure 3. The 
method 100 enables the tracking of the needle 60 even if the needle 60 is not 

25 coplanar and, thus, exits a 2D US image plane as a result of an oblique 

insertion. The method can also be used for the Identification, segmentation and 
tracking of needles if they are completely contained in a 2D US image plane. To 
perform near real-time needle segmentation for an oblique trajectory, capture of 
two 3D US images is required. A 3D US, image is comprised of two or more 2b 

30 US images that are offset. Note, that if the needle 60 is coplanar with a 2D US 
image, then two 2D US images can generally be used, but the procedure is 
unchanged. 
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[0036] The initial 3D US image is obtained by scanning the prostate 
(tissue) to obtain a set of 2D US images before the needle is inserted. This 3D 
US image establishes a baseline or control against which other images will be 
compared. A subsequent 3D US Image is then acquired by scanning only the 
5 region containing the needle. It is to be understood that the second 3D US 
image may not be, in fact, the next 3D US image captured after the first, but 
refers to any subsequently-captured 3D US image. The method, as described, 
is used to identify, segment and track the needle in each subsequent 3D US 
image captured after the first 3D US image is captured. Each new 3D US image 

10 is compared to the initial image to identify the position of the needle at that time. 
[0037] The method 100 commences with the performance of an initial 3D 

US scan (step 104). The needle 60 is then inserted into the target volume (step 
108). Next, a subsequent 3D US scan is performed (step 112). The needle 60 
Is segmented to distinguish its location using the initial and subsequent 3D US 

15 images (step 116). The needle trajectory is then determined (step 120). Once 
the needle trajectory has been determined, the needle tip and needle entry point 
locations within the reconstructed volume are determined (step 124). The 
needle tip and entry point locations are then reconstructed (step 128). An 
arbitrary third point in the target volume is selected (step 132). The plane 

20 defined by the needle tip and entry points and the arbitrary third point is 

extracted from the reconstructed 3D image (step 136). Next, the extracted 
plane is displayed (step 140). It is then determined if there are any remaining 
unanalyzed planes (step 144), If there are, the method 100 returns to step 132, 
at which another arbitrary point is selected. If, instead, all of the desired planes 

25 have been analyzed, the method 100 ends. 

[0038] During the performance of the initial 3D US scan at step 104, the 
MCM 44 and motor assembly 28 causes the TRUS transducer 24 to rotate 
about its long axis over about 100 degrees while image data corresponding to 
s 2D US images is captured at one degree intervals. The image data 

30 corresponding to the 2D US images is then transmitted to the computer 40 to be 
digitized by the video frame grabber 36 and registered by the imaging software. 
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[0039] The acquired 2D US Images are processed by the imaging . 
software as they are collected. The 2D US images correspond to planes radially 
extending from the central axis of rotation of the TRUS transducer 24. 
Accordingly, the 3D volume Is reconstructed by translating and rotating the 2D 
5 US images with respect to one another. The reconstructed 3D volume consists 
of an array of voxels, or 3D pixels. The voxels are typically cubic (but can also 
be rhomboidal) and are arranged according to a 3D Cartesian system. Each 
voxel is assigned a greyscale-level value based on the greyscale-level values of 
the pixels in the translated 2D images adjacent to it. 

10 [0040] Figure 4 illustrates a 3D US image reconstructed from the set of 

2D US images. As can be seen, the 3D US image has a fan profile 
corresponding to the volume Imaged by the TRUS transducer 24, The acquired 
2D US images are reconstructed into a 3D US image by the imaging software. 
The 3D visualization software then generates a view of the 3D US image,.and 

1 5 provides a multi-planar 3D display and volume rendering, as well as an 

extensive set of measurement tools. The 3D US image is then presented for 
viewing on the display of the computer 40. As each new 2D US Image is 
acquired by the TRUS transducer 24 during its rotation, the 3D visualization 
software dynamically updates the 3D image presented on the display. 

20 . [0041] During the performance of the subsequent 3D US scan at step 

1 12, a region of interest is identified, and the ultrasound imaging system 20 Is 
, focused on a segment of an operational scan range of the TRUS transducer 
encompassing the region of interest in a target volume. In particular, the TRUS 
transducer Is focused on the segment to capture images of the expected 

25 position of the needle 60. While the expected position of the needle 60 in the 
3D US Images can be determined based on the needle position coordinates 
provided by the positioning software, needle deviations in the 3D US Images 
can occur for a number of reasons. These include slight bending of the needle 
60 as it is inserted and shifting in the target volume. By obtaining a new 3D.US 

30 image, the actual position of the needle 60 can be more precisely determined. 
[0042] Figure 5 better illustrates the performance of the subsequent 3D 

US scan. The expected needle position is obtained from the positioning 
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software (step 210). The region of interest is determined based on the expected 
position of the needle, and a corresponding segment of the operational scan 
range of the TRUS transducer 24 is determined (step 220). Next, a scan 
strategy for the segment of the operational scan range is determined (step 230). 
5 In determining the scan strategy for the segment of the operational scan range 
at step 230, the positions of 2D US images to be acquired is determined. In 
particular, a set of 2D US images are planned at one-half degree intervals along 
the angular width of the scan region of interest. A scan Is then performed in 
accordance with the scan strategy (step 240). Data from the initial 3D US image 

10 is then used to complete the 3D US image (step 250). 

[0043] During the determination of the region of interest at step 220, the 

region of interest is selected to include the expected needle position obtained 
during step 210. Where the needle has yet to be inserted/detected, the region 
of interest is defined to be an area around the expected needle entry point. If, 

1 5 instead, the needle was at least partially inserted/detected at the time of the last 
3D US scan, the region of interest is determined to include the original needle 
position plus a distance along the needle trajectory beyond the needle tip as will 
be described. 

[0044] The region of interest is then reverse-mapped onto the operating 

20 coordinates of the TRUS transducer 24 and is used to determine a segment of 
the operational scan range of the TRUS transducer 24 that encompasses the 
region of interest at step 230. In particular, the segment of the operational scan 
range is selected to correspond to an angular sector of the operational scan 
range of the TRUS transducer 24 that encompasses the region of interest. 
25 Where the needle is inserted along an oblique trajectory and, consequently, 

intersects a number of 2D US images at points, the angular width of the sector is 
setected to sufficiently cover the region of interest plus five degrees of rotation to 
cover the distance along the needle trajectory beyond the needle tip. 
[0045] Figure 6 is an endrview of the TRUS transducer 24 and the 
30 segment of the operational scan range selected during step 220 for the needle 
when It is inserted along an oblique trajectory. A region of interest 280 
encompasses an expected needle position 282 and extends a distance past the 
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expected needle tip position 284. A segment of the operational scan range 288 
corresponding to the sector encompasses the region of interest 280. The 
segment of the operational scan range 288 includes a five-degree margin 292 to 
capture the region of Interest extending along the needle trajectory beyond the 
5 expected needle tip position 284. Two background areas 296 of the operational 
scan range of the TRUS transducer 24 flank either side of the sector. 
[0046] During the completion of the subsequent 3D US image at step 

250, data from the initial 3D US image is used to fill in the background areas. 
As the scan strategy can exclude the capture of some or all image data from the 
10 background areas, image data from the initial 3D US scan is used to fill in any 
image data required in the subsequent 3D US image. The image data in the 
background areas is not expected to change and can, thus, be borrowed from 
the initial 3D US image. 

[0047] By modifying the behavior of the TRUS transducer 24 to focus on 

1 5 t\\e region of interest, more detailed information can be captured around the tip 
of the needle 60 on a near real-tinie basis. Further, by reducing the scanning 
density for the other areas, the additional time required to scan the region of 
interest can be compensated for. 

[0048] After the initial and subsequent 3D US scans have been 
20 completed, the needle 60 is segmented at step 116. The subsequent 3D US 

image is compared to the initial 3D US image, and the needle position within the 
subsequent 3D US image, including the needle tip and entry point location, is 
determined. The needle 60 will show up as voxels with a greyscale-level 
change that exceeds a threshold value between the initial and subsequent 3D 
25 US images. There can be, however, other voxels with a greyscale-level change 
that exceeds the threshold value that do not. In fact, represent the needle, but 
may represent, for example, calcifications in the prostate. In order to permit 
better Identification of the actual needle, the system 20 attempts to Identify and 
discard these other voxels. 
30 [0049] Figure 7 better illustrates the method of needle segmentation at 
step 116. The method commences with the calculation of a greyscale-Ievel 
change threshold (step 310). A difference map is then generated from the initial 
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and subsequent 3D US images (step 320). Next, the difference map Is pre- 
filtered (step 330). Regression analysis is performed on the difference map to 
identify the needle (step 340). The result of the regression analysis Is then 
analyzed to detennine if it is satisfactory (step 350). If the results are 
5 detemnlned to be unsatisfactory, the difference map is filtered (step 360), and 
the method returns to step 340, where regression analysis is again performed 
on the filtered image. The filtering of the difference map and the regression 
analysis is repeated until all of the voxels in the difference map are within a 
prescribed range from the regression line. As the filtering removes outlying 

10 voxels, their effect on the linear regression is removed, thereby allowing the 
needle trajectory to be more accurately estimated. Reiterative filtration of the 
difference map is performed to obtain a desired level of confidence in the 
estimated needle trajectory. Once the result of the regression analysis Is 
deemed to be satisfactory at step 350, the method ends, >v: 

1 5 [0050] Figure 8 better illustrates the calculation of the greyscale-level 

change threshold at step 310. A greyscale-level change threshold value, GLC 
threshold, is used to reduce the number of voxels to be analyzed in the 3D US 
images and to obtain candidate needle voxels. To determine the threshold 
value, the maximum greyscale-level value, GL^ax, in the subsequent 3D US 

20 image is first determined by examining each voxel in the image, and then GLmax 
is multiplied by a constant. 

[0051] The calculation of GLC threshold commences with the setting of 

GLmax io zero (step 410). A voxel is then selected from the subsequent 3D US 
image (step 420). The greyscale-level value, GLvafue, of the selected voxel is 

25 determined (step 430). The greyscale-level value of the selected voxel, GLvaiuof 
is then compared to the maximum greyscale-level value, GLmax (step 440). If 
the greyscale-level value of the selected voxel, GLvaiuef is greater than the 
maximum greyscale-level value, GLmax. the value of GLmax is set to GLvaJu& (step 
450). It is then determined whether there are any unanalyzed voxels remaining 

30 in the subsequent 3D US image (step 460). If there are, the method returns to 
step 420, where another voxel is selected from the subsequent 3D US image. 
If, instead, it is determined at step 460 that there are no remaining unanalyzed 
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voxels in the subsequent 3D US image, the greyscale-level change threshold 
value is calculated as follows: 

GLC threshold = a x GLmax (Eq. 1) 

where 0 < a < 1 . A value for a of 0.5 provides desirable results. 
5 [Q052] Figure 9 better illustrates the generation of a difference nnap 

during step 320 using the threshold calculated during step 310. The difference 
map is a registry of candidate needle voxels that represent an area of the same 
size as the initial and subsequent 3D US images. Initially, the greyscale-level 
value of each voxel in the initial 3D US image is compared to that of its 
10 counterpart in the subsequent 3D US image, and the difference Is determined: 

GLC(iJM) = postGL(ij\k) - preGL(ij\k) (Eq. 2) 

where preGL(iJ,k) and postGL(ij)k) are the greyscale-level values of voxels at 
location mk) in the initial and subsequent 3D US images respectively, and 
GLC(ij\k) is the greyscale-level change. 
1 5 [0053] Those voxels in the subsequent 3D US image whose grayscale- 

level values exceed those of their counterpart in the initial 3D US image are 
deemed to have changed significantly and are registered in the difference map. 
That is, 

(U Jm, km) e 3D DM, where GLC(imJm.krn) > GLC threshold (Eq. 3) 

20 for m = 1 , 2, ■ . n, where n is the number of points included in the 3D difference 
map. The remaining voxels having greyscale-level values that do not exceed 
those of their counterpart in the initial 3D US image are deemed to have 
changed insignificantly and are not added to the difference map. 
[0054] The method of generating the difference map begins with the 

25 selection of a voxel in the subsequent 3D OS image and its counterpart in the 
initial 3D US image (step 510). The greyscale-level difference, GLdiff, between 
the voxels of the initial and subsequent 3D US Images is found (step 520). The 
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greyscale-level difference, GLd/ff, is compared to the greyscale-Ievel change 
threshold, GLC threshold, to determine if it exceeds it (step 530). If it is 
determined that the greyscale-Ievel difference, GLdiff, exceeds the greyscale- 
level change threshold, GLC threshold, the position of the voxel is added to the 
5 difference map (step 540). It is then determined whether there are any 

remaining unanalyzed voxels in the initial and subsequent 3D US images (step 
550). If it is determined that there are unanalyzed voxels remaining in the initial 
and subsequent 3D US images, the method returns to step 510, where another 
pair of voxels is selected for analysis. If, instead, it is determined that all of the 
10 voxels in the initial and subsequent 3D US images have been analyzed, the 
method of generating the difference map ends. 

[0055] During pre-filtration of the difference map at step 330, voxels 
registered In the difference map are analyzed to remove any voxels that are 
deemed to be noise. In the system 20, the 3D Image is advantageously . 
1 5 reconstructed orTdemancI and, therefore; access to the^original acquired Image 
data is available. 

[0056] Voxels are identified and analyzed to determine whether they 

correspond to a characteristic of the needle. Since the image of the needle is 
expected to extend along the 3D scanning direction, voxels representing the 
20 needle are assumed to be generally adjacent each other along this direction. 

Other voxels in the difference map that are more than a pre-determined distance 
along this direction from other voxels are deemed to be noise and removed. 
That is, assuming that k is the direction along which the needle is expected to 
extend, voxels are removed from the difference map as follows: 

25 (ImsJm, km) 0 3D DM, where ^GLC (imjm,km±s) < GLC threshold (Eq. 4) 

where, s = 1 , 2,. . and P is the number of voxels sun-oundlng voxel (/m, Jm, 
km) in the /c-direction. A value for P of 4 provides desirable results. 
[0057] Figures 1 0a and 1 0b show the difference map prior to and after 
pre-filtration respectively. As can be seen, spurious voxels not occurring in 
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clusters extending along the same path as the needle are removed during pre- 
filtration. 

[0058] Once the difference map has been pre-filtered, regression 
analysis is performed on the difference map at step 340. During this analysis, a 
5 line is fit to the voxels in the difference map using linear regression analysis. 
The equation of the line determined from the difference map using linear 
regression analysis provides the estimated trajectory for the needle. 
[0059] Figure 1 1 better illustrates the performance of the regression 

analysis on the difference map at step 340, A voxel registered in the difference 

10 map is selected (step 610). The volume is projected along the z-axis to find a 
first trajectory (step 620). Next, the volume is projected along the y-axis to find a 
second trajectory (step 630). It is then determined if there are any unanalyzed 
voxels in the difference map (step 640). If it is determined that there are 
unanalyzed voxels in the difference map, the method returns to step 610, where 

1 5 another vojcel is selected in the difference map for analysis. If, instead, all of the 
voxels in the difference map have been analyzed, the results of the first 
trajectory are used to obtain y and the results of the second trajectory are used 
to obtain z, given x (step 650). Once (x,y,z) has been determined, the method 
240 ends. 

20 [0060] If it is determined at step 350 that the linear regression is 

unsatisfactory, the difference map is filtered at step 360. 
[0061] Figure 12 better illustrates the filtering of the difference map. 

During the filtering of the difference map, spurious voxels that are further than a 
pre-determined distance from the estimated trajectory of the needle determined 

25 during step 340 are removed. 

[0062] The method of filtering the difference map commences with the 
selection of a voxel in the difference map (step 710). The distance to the 
estimated needle trajectory Is measured in voxels (step 720). A determination is 
then made as to whether the distance between the voxel and the estimated 

30 needle trajectory is greater than a pre-determined distance limit (step 730). It 
has been found that filtering out voxels further than five voxels in distance from 
the segmented needle trajectory provides desirable results. If the distance 
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determined is greater than the pre-determlned distance limit, the voxel is 
removed from the difference map (step 740). Then, it is determined If there are 
any unanalyzed voxels remaining in the difference map (step 750). If there are, 
the method retums to step 710, wherein another voxel in the difference map is 
5 selected for analysis. If, instead, all of the voxels in the difference map have 
been analyzed, the method of filtering the difference map ends. 
[0063] Figure 13 shows the difference map of Figures 10a and 10b after 
filtration at step 360 and immediately prior to the final regression calculation. As 
can be seen, the difference map is free of spurious voxels distant from the 

10 visible needle trajectory. 

[0064] As mentioned previously, once the needle trajectory has been 
determined, the needle entry point and needle tip locations are reconstructed at 
step 124. The needle entry point Is detennlned to be the Intersection of the 
needle trajectory and the known entry plane. The needle tip is deemed to be 

1 5 the furthest needle voxel along the needle trajectory. 

[0065] After the needle tip and entry point have been reconstructed, an 
arbitrary third point in the subsequent 3D US image is selected at step 128. To 
extract any plane containing the needle, the segmented needle entry point, 
needle tip point and a third point within the subsequent 3D US image are used 

20 to define a specific plane that is coplanar with the needle (i.e., contains the 

needle lengthwise). The location of the arbitrary point determines whether the 
plane will be sagltal-oblique or coronal oblique. For a sagltal-oblique plane, the 
arbitrary point is picked on a line going through the needle entry point and 
parallel to the y-axis. For a coronal-oblique plane, the arbitrary point is picked 

25 on a line going through the needle entry point and parallel to the x-axis. 
[0066] The data occurring along the plane in the 3D US image Is 
extracted at step 132 to permit generation of a 2D US Image of the plane. In 
this way, the oblique saggital, coronal and transverse views with the needle 
highlighted can be extracted and displayed. . 

30 [0067] Once the plane is extracted, the 2D US image of the plane is 
presented on the display of the computer 40 at step 136. The location of the 
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needle 60 in the 2D US image is demarcated using a colored line in tlie 
greyscale image to facilitate visual identification of the needle. 
[0068] It is then determined whether there remain any unanalyzed planes 
at step 140. As three planes are displayed by the computer 40 at the same 
5 time, the process is repeated twice to obtain the other two planes. The first 
plane selected for analysis is the saggital plane and the other two planes are 
orthogonal to the first plane. If there are, the method returns to step 128, where 
another arbitrary point is selected to define another plane. Otherwise, the 
method 100 ends. 

10 [0069] Figures 14a to 14c show a 2D US image obtained using the 

method 100 during a patient's prostate cryotherapy procedure, demonstrating 
that the needle can be tracked as it is being inserted and orthogonal views can 
be displayed for the user during the insertion procedure. 

Evaluation 

15 Experimental apparatus 

[0070] The accuracy and variability of the needle segmentation and 

tracking technique was tested using images acquired by scanning phantoms. 
Referring again to Figure 1 , the robot 48 shown was used to insert the needle 
60 at known angles, including oblique trajectories with respect to the TRUS 
20 image plane. 

[0071] The needle used in these experiments was a typical 18-gauge 

(i.e., 1.2mm in diameter) prostate brachytherapy needle. The two US tissue- 
mimicking phantoms were made of agar, using a recipe developed by D.W.. 
Ricky, P.A. Picot, D.C. Christopher, A. Fenster, Ultrasound Medical Biology, 
25 27(8), 1025-1034, 2001, and chicken breast tissues. TRUS images were 
obtained using an 8558/S side-firing linear array transducer with a central 
frequency of 7.5MHz, attached to a B-K Medical 2102 Hawk US machine (B-K, 
Denmark). The computer was a Pentium III personal computer equipped with a 
Matrox Meteor II video frame grabber for 30Hz video image acquisition. 
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Algorithm execution time 

[0072] Execution time is dependent on the 3D scanning angular interval 

and the. extent of the region to be investigated. To evaluate the execution time 
of the disclosed method of needle segmentation the initial 3D US scan was 
5 performed, and then the needle was inserted. After needle insertion, the 

phantom was scanned again, and the needle was segmented. A software timer 
was used to measure the time elapsed during the execution of the 
segmentation. 

Accuracy test 

1 0 [0073] To test the accuracy of the method, the robot was used to guide 

the needle insertion into the phantom at known angles. The angulation 
accuracy of the robot was evaluated^to be 0.12 ± 0.07 degrees. 
[0074] First, the robot was used to guide the needle insertion along a 

trajectory parallel to the TRUS transducer 24, hereinafter referred to as the zero 

1 5 (0) degree orientation. Since the needle could be verified by observing the 

needle in the real-time 2D US image, this trajectory was assumed to be correct. 
As a result, oblique trajectory accuracy measurements could be made with 
respect to the zero degree trajectory. The positions of the needle tip and the 
needle entry point were then found for the zero degree trajectory using the 

20 method described above. The robot 48 was used to insert the needle at 

different angles (+5, +10, +15, -5, -10 and -15 degrees) with respect to the zero 
degree trajectory. For each insertion, the positions of the needle tip and the 
needle entry point were found. The corresponding segmented needle vectors 
through the needle entry point and needle tip were determined by using the 

25 following fonnuia: 

costal. =M (Eq.5) 

m 

where A Is the segmented needle vector for the zero degree trajectory; b is the 
segmented needle vector for the insertion at any other angle; 0^^^ is the angle 
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derived from the segmentation algorithm. The accuracy of the algorithm was 
evaluated by comparing e^^^ with the robot orientation angle . The error, sq, 

was determined as follows: 

^.=Kig-^rob| (Eq.6) 

5 [0075] The accuracy test was repeated with a chicken tissue phantom, 
and the accuracy was again determined using Equations 5 and 6. For the agar 
phantoms, five groups of tests were performed to evaluate the algorithm 
execution time and accuracy. Each group consisted of seven insertions; Ae., 
insertion at 0, +5, +10, +15, -5, -10 and -15 degrees. The mean error as a 
1 0 function of insertion angle, €e, was calculated as follows: 

m = ^ ^ — (0.7) 

Results and Conclusion 

[0076] The following table presents the evaluation results. In the chicken 
tissue phantom, the average execution time was 0.13 ± 0.01 seconds, and the 
1 5 average angulation error was 0.54 ± 0.1 6 degrees. In agar phantoms, the 

average execution time was 0.12 ± 0.01 seconds, and the average angulation 
error was 0.58 ± 0.36 degrees. The results shown below also demonstrate that 
the insertion error does not significantly depend on insertion angle. 



Angle (degrees) 





-15 


-10 


-5 


+5 


+10 


+15 


1 


Time 
(seconds) 


0.13 


0.11 


0.12 


0.12 


0.12 


0.14 




Accuracy 
(degrees) 


0.50 


0.51 


0.43 


0.37 


0.74 


0.74 


2 


Time 
(seconds) 


0.12 


0.12 


0.12 


0.11 


" 0.12 


0.13 




Accuracy 
(degrees 


0.30 


0.71 


0.48 


0.68 


0.42 


0.86 



20 



wo 2005/067800 



PCT/CA2005/000032 



- 22 - 

[0077] In 3D US images, needle voxels generally have high greyscale- 
levei values. However, due to specular reflection, some background structures 
may also appear to have high greyscale-level values. This increases the 
difficulty in automatic needle segmentation in a US image using greyscaie-Ievel 
5 information directly. As US images suffer from low contrast, signal loss due to 
shadowing, refraction and reverberation artifacts, the greyscale-level change 
detection technique of the disclosed embodiment of the invention appears to be 
quite robust In addition, since the needle is segmented from a difference map, 
complex backgrounds can be ignored to simplify calculations and accuracy. 

10 [0078] In conclusion, a greyscale-level change detection technique has 

been developed and its feasibility has been tested for near real-time oblique 
needle segmentation to be used in 3D US-guided and robot-aided prostate 
brachytherapy. The results show that the segmentation method works well in 
agar and chicken tissue phantoms. In addition, the approach has also been 

1 5 tested during several prostate cryotherapy procedures with positive results. 

Alternative IVIethods of Defining the Region of Interest and Scan 
Strategies 

[0079] A number of alternative methods for defining the region of interest 

and scan strategies have been explored for use with the system 20. In a first 

20 alternative, the region of interest is defined to include only a set length of the 
needle from the tip plus a pre-determined distance beyond the needle tip along 
the needle trajectory. For example, the region of interest can be defined to 
include a one-half-inch length of the needle measured from its tip and an area 
one-half inch along its trajectory beyond the needle tip. The scan, strategy then 

25 is selected to capture 2D US images at one-half degree intervals along the 

angular width of the segment of the operational scan range of the transducer of 
the ultrasound imaging system encompassing the region of interest As the 
needle is further inserted into the target volume, the region of interest roams 
with the needle tip. Using this approach, 2D US images can be rapidly captured 

30 and updated to provide accurate information about the position of the needle tip. 
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[0080] In another alternative method for defining the region of interest 

and scan strategy, the region of interest is defined to include an area of 
expectied activity of a one-half-inch length of the needle measured from Its tip 
and an area one-half inch along its trajectory beyond the needle tip. This area 
5 of expected activity generally allows the new position of the needle to be 

determined when compared to previous images. A scan strategy can then be 
selected to scan a segment of the operational scan range of the transducer of 
the ultrasound imaging system encompassing the region of interest using a fine 
scan density, and other areas using a coarse scan density. By selecting a 

10 relatively high scan density for the subset of the operational scan range of the 
transducer of the ultrasound imaging system and a relatively low scan density 
for other scan areas (e.g. one 2D US image every one-half degree Interval in the 
region of interest, and every one-and-one-half degree Interval outside the region 
of interest), detailed information about the region of interest can be obtained 

1 5 while still capturing a desired minimum level of detail about other areas. 
[0081] Where the needle has yet to be detected, and information 

regarding the expected needle entry point is available, the region of interest can 
be defined to include an area surrounding the expected needle entry point. 
[0082] Where the needle is not determined to be present in the region of 

20 interest, additional 2D images can be acquired to locate the needle. 

[0083] Other alternative methods for defining the region of interest and 

scan strategy and combinations thereof will occur to those skilled in the art. 
[0084] While the method of registering the position of an object moving in 
a target volume in an ultrasound imaging system and the method of imaging 

25 using an ultrasound imaging system have been described with specificity to a 
rotational US scanning method, other types of scanning methods will occur to 
those of skill in the art. For example, the same approach can be used with a 
linear US scanning method. In addition, the segmentation method can be 
applied equally well to 3D US images reconstructed using the linear scanning 

30 geometry, but acquired using rotational 3D scanning geometry such as that 
. used in prostate imaging. 
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[0085] The linear regression analysis approacfi for determining the 
needle trajectory from the difference map was selected as it requires relatively 
low processing power. A person of skill in the art, however, will appreciate that 
any method of determining the needle trajectory given the difference map can 
5 be used. For example, the well-known Hough Transform technique can be 
employed. The Hough Transform technique requires higher computational 
power than the linear regression approach, but.this can be ignored where such 
processing power is available. 

[0086] While a specific method of determining the GLC threshold was 

1 0 disclosed, other methods of determining the GLC threshold will occur to those 
skilled in the art. For example, a histogram of the greyscale-level values in the 
30 US image can be generated and then analyzed to determine the regions of 
the histogram that most Ijkeiy correspond to the background and to the needle. 
The analysis can be based on the statistical distribution of the greyscale-level 
1 5 values due to the acoustic scattering of the tissue and the statistical distribution 
of the specular reflection of the needle. 

[0087] In addition to 3D applications, difference maps can be used to 

register movement in a single 2D plane. In this case, the difference map could 
represent a 2D plane and register differences between two 2D images. 

20 [0088] While, in the above-described embodiment, the expected needle 

position from the positioning software was used to determine the region of 
interest thereby to modify the scanning behavior of the TRUS transducer 24, 
one or more previous images could be used to estimate the expected needle 
position. For example, where only the immediately previous image is available, 

25 the region of interest could include the needle plus a relatively large distance 
along its trajectory beyond the needle tip. Where two previous images are 
available, the region of interest could include the needle plus a distance along its 
trajectory beyond t.he needle tip, wherein the distance is determined from 
movement of the needle registered from the two previous images. 

30 [0089] While, in the described embodiment, an object of interest in the 

ultrasound images is a rieedle, those skilled in the art will appreciate that the 
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invention can be used in conjunction with other objects, such as, for example, 
biopsy apparatus. 

[0090] It can be advantageous in some cases to compare a US image to 

one or more previous US images. For example, where the target volume is 
5 expected to shift, the initial image of the target volume prior to insertion of the 

needle may provide an inaccurate baseline image. By using more recent 
previous images, the target volume can be, in some cases, more readily filtered 
out to generate a cleaner difference map. 

[0091] While the US images are pre-filtered to identify voxels that are 

10 adjacent other voxels along the expected direction that the needle longitudinally 
extends, other methods of filtering the images will occur to those skilled in the 
art. Voxels corresponding to other characteristics of an object can be identified 
to filter out other voxels that do not correspond to the same. 
[0092] The above-described embodiments are intended to be examples 
15 of the present invention and alterations and modifications may be effected 
thereto, by those of skill in the art, without departing from the scope of the 
invention which is defined solely by the claims appended hereto. 



